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NOTICE 

This  document  is  disseminated  under  the  sponsorship 
of  the  U.S.  Department  of  Transportation  in  the  interest  of 
information  exchange.  The  United  States  Government 
assumes  no  liability  for  the  contents  or  use  thereof. 

The  United  States  Government  does  not  endorse 
products  or  manufacturers.  Trade  or  manufacturers’ 
names  appear  herein  solely  because  they  are  considered 
essential  to  the  objective  of  this  report. 
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EXECUTIVE  SUMMARY 


The  Federal  Aviation  Administration  (FAA)  is  currently  evaluating  low  rate  voice 
digitizing  coder/decoder  (CODEC's)  for  use  with  the  Aeronautical  Mobile  Satellite 
Service  (AMSS).  Phase  II  of  this  evaluation  consisted  of  air  traffic  control 
(ATC)  personnel  participating  in  an  objective  intelligibility  test  of  several 
CODEC's  under  operational  conditions.  The  results  of  the  testing  show  that  the 
intelligibility  of  the  low  rate  4.8  kilobits  per  second  (kbps)  CODEC  is 
essentially  equivalent  to  the  intelligibility  of  the  9.6  kbps  CODEC.  The  results 
also  show  that  the  4.8  kbps  CODEC's  can  operate  with  high  intelligibility  under 
conditions  of  high  bit  error  rates  and  operational  background  noise. 
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1.  INTRODUCTION. 


1.1  OBJECTIVE. 

The  primary  objective  of  the  Phase  II  coder/decoder  (CODEC)  testing  is  to 
evaluate  the  voice  intelligibility  and  acceptability  of  low  data  rate  voice 
digitizing  CODEC's  for  use  in  air  traffic  control  (ATC) . 

1.2  BACKGROUND. 

The  International  Civil  Aviation  Organization  (ICAO)  has  developed  requirements 
for  the  Aeronautical  Mobile  Satellite  Service  (AMSS)  for  use  by  the  civil 
aviation  community;  satellite  systems  are  currently  under  development  to  meet 
these  needs.  In  the  1990's,  such  systems  will  provide  data  and  digitized  voice 
communications  with  aircraft  for  ATC,  airline  operations  and  administration,  and 
passenger  use. 

Testing  was  conducted  by  the  Airline  Electronics  Engineering  Committee  (AEEC)  to 
select  a  9.6  kilobits  per  second  (kbps)  CODEC  for  passenger  telephony  over  AMSS. 
The  Federal  Aviation  Administration  (FAA)  is  continuing  to  evaluate  lower  data 
rate  CODEC's  for  suitability  in  the  ATC  environment. 

The  purpose  of  evaluating  the  intelligibility  of  low  data  rate  voice  CODEC's  is 
to  determine  their  suitability  for  use  in  the  ATC  environment.  The  9.6  kbps 
CODEC's  are  generally  accepted  as  practical  for  voice  communications.  However, 
the  lower  data  rate  CODEC's,  if  found  to  be  suitable,  offer  system  performance 
advantages  due  to  their  lower  radio  frequency  (RF)  transmission  power  and 
bandwidth  requirements.  The  technology  of  lower  rate  CODEC's,  especially  those 
operating  at  transmission  rates  of  4.8  kbps,  is  rapidly  improving.  With  improved 
processing  algorithms,  some  CODEC's  at  these  lower  rates  may  be  acceptable  for 
ATC  use. 

The  FAA  completed  a  subjective  CODEC  evaluation  in  Phase  I  of  its  CODEC  testing 
program  in  which  ATC  personnel  from  the  New  York  Center  listened  to  recordings 
of  voice  messages  processed  by  the  various  CODEC's  under  test.  The  controllers 
gave  their  opinion  of  voice  acceptability  and  intelligibility,  with  Mean  Opinion 
Scores  (MOS)  ranging  from  0  for  "bad"  to  5  for  "excellent." 

The  Phase  I  tests  were  conducted  under  quiet  background  noise  conditions.  In 
preparing  the  audio  test  tapes,  the  voice  messages  were  processed  through  CODEC's 
or  CODEC  software  algorithms  without  inducing  any  transmission  bit  errors.  The 
test  results  indicated  that  the  intelligibility  and  acceptability  of  the  best 
4.8  kbps  CODEC's  were  within  0.5  MOS  points  of  the  9.6  kbps  CODEC. 

The  results  of  Phase  I  testing  demonstrated  that  the  performance  of  4.8  kbps 
CODEC's  was  close  to  that  of  9.6  kbps  CODEC's.  However,  Phase  I  included  only 
subjective  opinion  testing;  no  background  noise  or  bit  errors  were  introduced  to 
simulate  actual  field  operational  conditions.  Phase  I  testing  concentrated  on 
opinion  score  testing  of  CODEC's  to  evaluate  their  intelligibility  performance. 
Phase  II  of  the  test  program,  built  on  the  results  of  Phase  I,  focused  on  more 
objective  intelligibility  tests,  as  opposed  to  subjective  opinion  scoring,  and 
used  only  CODEC's  available  in  hardware  form. 
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The  principal  intent  of  Phase  II  was  to  develop  objective  intelligibility  test 
procedures  meaningful  for  the  ATC  environment  and  to  measure  the  intelligibility 
and  acceptability  of  several  CODEC's  using  those  procedures. 

1.3  RELATED  DOCUMENTATION. 

a.  International  Civil  Aviation  Organization  (ICAO),  Report  of  the  Fourth 
Meeting  of  the  Special  Committee  on  Future  Air  Navigation  Systems  fFANS/4) . 
Montreal,  1988. 

b.  Aeronautical  Radio,  Inc.  (ARINC) ,  Aviation  Satellite  Communications 
System.  Part  2.  System  Design  and  Equipment  Functional  Description.  ARINC 
Characteristic  741 .  Annapolis,  March  15,  1989. 

c.  Airlines  Electronic  Engineering  Committee  (AEEC) ,  Report  of  Satellite 
Subcommittee's  Voice  Coding  Working  Group  Meetins  Held  December  13.  1988,  in 
Arlington.  Virginia.  AEEC  Letter  89-006/SAT-96 ,  February  13,  1989. 

d.  Child,  J.,  Cleve ,  R. ,  and  Grable,  M. ,  Evaluation  of  Low  Data  Rate  Voice 
CODECS  for  Air  Traffic  Control  Applications,  Technical  Note,  CTA  Inc.  for  the 
FAA  Technical  Center,  D0T/FAA/CT-TN89/13,  January  1989. 

2.  TEST  PROGRAM. 

2.1  TEST  OBJECTIVES 

The  test  objectives  of  Phase  II  CODEC  testing  were: 

a.  To  develop  objective  test  procedures  to  evaluate  the  intelligibility  of 
CODEC's  in  an  AT C  environment. 

b.  To  use  these  procedures  to  evaluate  the  intelligibility  of  low  bit  rate 
CODEC's  (4.8  and  2.4  kbps)  available  in  hardware  to  determine  if  low  data  rate 
CODEC  technology  is  mature  enough  for  application  in  the  ATC  environment 

c.  To  quantify  the  effects  of  bit  error  rate  and  background  noise  on  CODEC 
intelligibility. 

d.  To  determine  the  acceptability  of  the  CODEC's  under  test  by  collecting 
the  opinions  of  the  ATC  personnel  and  to  relate  this  subjective  opinion  to  the 
more  objective  intelligibility  results. 

e.  To  conduct  these  evaluations  over  simulated  telephone  lines. 

2.2  TEST  METHODOLOGY. 

Normally,  an  intelligibility  test  involves  people  listening  to  recordings  of 
aural  messages.  The  ability  of  the  listener  to  properly  understand  the  messages 
is  a  measure  of  intelligibility.  There  are  many  different  forms  of  test  message 
texts  and  methods  of  scoring  intelligibility.  Intelligibility  test  texts  can  be 
classified  in  terms  of  their  "openness."  A  "closed"  text  is  a  set  of  messages 
that  are  well  known  to  the  listener  or  have  a  limited  number  of  possible 
interpretations.  For  example,  the  listener  may  be  asked  to  choose  between  two 
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words  such  as  "show"  and  "snow."  An  example  of  such  a  "closed"  test  is  the 
industry  standard  Diagnostic  Rhyme  Test  (DRT) . 

An  "open"  test  consists  of  text  with  a  large  number  of  possible  interpretations 
by  the  listener.  An  example  of  "open"  text  is  a  list  of  polysyllabic  nonsense 
words.  When  using  such  text,  there  is  a  very  large  number  of  possible 
interpretations  by  the  listener. 

Other  intelligibility  tests  fall  between  the  "closedness"  of  the  DRT  and  the 
"openness"  of  nonsense  text.  Specialized  intelligibility  tests  are  often 
desirable  to  help  determine  the  suitability  of  a  voice  system  for  specific 
applications.  In  some  circumstances,  voice  messages  are  normally  spoken  in  a 
partially  "closed"  environment.  For  example,  when  talking  to  an  ATC  Center, 
aircraft  pilots  normally  use  a  limited  vocabulary.  Almost  all  their  words  are 
spoken  in  a  context  that  is  well  known  to  the  listener.  This  enhances  the 
intelligibility  of  words  in  the  messages,  resulting  in  higher  intelligibility 
than  would  be  experienced  testing  with  nonsense  syllables.  Likewise,  the  pilot's 
messages  will  be  less  "closed"  than  a  DRT  text. 

To  evaluate  the  performance  of  a  CODEC  in  the  ATC  environment,  tests  were  run  at 
three  levels  of  "openness."  Part  1  of  the  test  is  a  "closed"  pseudo-DRT  test. 

It  differs  from  the  industry  standard  DRT  test  in  that  some  of  the  word  pairs  of 
Part  1  are  polysyllabic.  Also,  a  number  of  words  common  to  the  ATC  environment 
were  included  in  the  text  without  regard  to  the  phonetic  balance  of  the  resulting 
word  set.  Although  Part  1  differs  from  standard  DRT  tests,  it  is  "closed"  and 
allows  the  overall  test  results  to  be  equated  to  generally  accepted 
intelligibility  metrics. 

Part  2  consists  of  pilot-ATC  Center  messages  wherein  the  test  listener  identifies 
spoken  numerals  within  sentences.  For  example,  the  message  may  be  "United  flight 

62  requesting  weather  . "  The  listener  is  to  identify  the  number  62.  Part  2 

is  considered  a  partially  "open"  test. 

The  text  for  Part  3  consists  of  isolated,  unrelated  words.  Some  of  the  words, 
such  as  "pilot,"  are  commonly  used  in  the  ATC  environment.  Therefore,  Part  3  is 
not  fully  "open"  since  such  words  are  well  known  and,  to  an  extent,  anticipated 
by  the  listener.  By  using  such  words,  the  test  provides  some  understanding  of 
the  utility  of  each  CODEC  in  the  ATC  environment. 

Test  sheets  used  for  all  three  parts  are  shown  in  appendix  A. 

To  simulate  real-world  ATC  conditions,  some  of  the  voice  messages  used  for  the 
tests  were  subjected  to  degradation  such  as  background  noise.  Further,  the 
processed  voice  signals,  in  their  digital  format,  were  exposed  to  bit  errors. 

This  process  simulates  corruption  of  the  signal  within  a  transmission  system 
carrying  the  signal. 

The  background  noise  conditions  tested  were: 

a.  Quiet  (no  additional  noise  added). 

b.  ATC  room  noise  (recorded  at  an  ATC  Center;  average  noise  level  was 
73  decibels  adjusted  (dBa) . 
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c.  Cockpit  noise  (recorded  in  a  Boeing  727  cruising  at  14,500  feet;  average 
noise  level  was  83  dBa) . 

The  bit  error  rates  (BER)  tested  were: 

a.  1x10'-’,  which  is  considered  achievable  under  normal  circumstances. 

b.  Ixl0'3,  which  is  considered  the  normal  operational  BER. 

c.  1x10' 2,  which  is  considered  a  worst  case.  Many  CODEC's  cannot  maintain 

synchronization  at  error  rates  above  this  error  rate. 

The  three  noise  and  three  BER  conditions  were  randomly  distributed  over  the  test 
messages  in  all  three  parts  of  the  Phase  II  tests.  Therefore,  all  CODEC's  were 
evaluated  under  nine  different  conditions  of  noise  and  error  rate  in  each  part. 

Each  of  the  three  parts  contained  36  test  messages.  They  were  divided  among  the 
nine  possible  conditions,  providing  four  messages  under  each  of  the  nine 
conditions . 

A  second  three-part  set  of  test  messages  was  generated.  It  consisted  of 
essentially  the  same  text  as  the  first  set  with  slight  variations,  such  as 
reversal  of  word  pairs  in  the  DRT  test.  These  changes  were  intended  to  help 
prevent  listeners  from  memorizing  the  messages  during  one  CODEC  test  and 
transferring  the  knowledge  to  the  next  CODEC  tested.  The  sets  of  messages  are 
referred  to  as  Script  1  and  Script  2.  The  combinations  of  script,  message, 
noise,  and  BER  are  shown  in  appendix  B. 

In  addition  to  the  intelligibility  test,  the  ATC  personnel  were  also  asked  to 
fill  in  a  questionnaire  regarding  the  acceptability  of  the  CODEC  under  test  and 
to  compare  the  CODEC  to  the  current  voice  communications.  The  questions  are 
included  at  the  end  of  the  test  sheets  and  are  shown  in  appendix  A. 

2.3  TEST  TAPE  GENERATION. 

To  generate  the  message  recordings  for  the  CODEC  testing,  the  messages  were 
spoken  by  eight  volunteer  ATC  controllers  from  the  New  York  Air  Route  Traffic 
Control  Center  (ARTCC)  and  four  pilot  volunteers  from  the  FAA  Technical  Center, 
Atlantic  City  International  Airport,  N.J.  The  volunteers  first  recorded  test 
message  text  with  a  quiet  background.  This  was  followed  with  recording  the 
messages  with  background  noise  (ATC  room  or  cockpit  noise)  played  at  the 
appropriate  level  in  the  same  room. 

The  recordings  were  edited  to  produce  two  master  recordings  corresponding  to 
Script  1  and  Script  2.  The  master  recordings  were  then  played  through  a  phone 
line  simulator,  which  was  configured  to  model  an  analog  telephone  link,  and  then 
digitized  by  each  CODEC.  The  digitized  voice  was  then  degraded  by  injecting  bit 
errors  using  a  satellite  channel  simulator.  The  digitized  voice  signal  was  fed 
to  the  receive  port  of  the  CODEC.  The  resulting  reconstituted  voice  output  was 
recorded  on  an  audio  cassette.  The  facility  for  generating  these  tapes  is 
discussed  in  detail  in  appendix  C.  This  process  was  repeated  for  each  of  the 
CODEC's  under  test.  A  total  of  11  CODEC's  were  tested;  one  at  9.6  kbps,  five  at 
4.8  kbps,  and  five  at  2.4  kbps.  Further,  unknown  to  the  listeners,  the 
original,  unprocessed  voice  scripts  were  evaluated  as  a  means  of  control  for  the 
testing . 
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The  tested  CODEC's  were  obtained  under  loan  from  their  manufacturers.  As  part  of 
the  loan  arrangement,  it  was  agreed  that  the  test  results  would  not  be  publicly 
associated  with  any  specific  CODEC.  During  this  testing,  the  CODEC's  are 
identified  only  by  the  letters  A  -  M  (CODEC  F  is  actually  clear  voice,  not 
processed  through  a  CODEC).  The  CODEC  algorithms  and  data  rates  are  shown  in 
table  1. 

2.4  TEST  ADMINISTRATION. 

Both  ATC  personnel  and  non-ATC  personnel  took  part  in  the  testing.  The  non-ATC 
personnel  were  CTA  Incorporated  office  staff,  and  the  ATC  personnel  were 
controllers  from  the  Washington  ARTCC  in  Leesburg,  Virginia,  and  the  Oakland 
ARTCC  in  California.  In  order  to  minimize  the  time  required  for  the  ATC 
personnel,  only  6  of  the  original  11  CODEC's  were  included  in  the  ATC  portion  of 
the  testing.  The  non-ATC  personnel  first  evaluated  the  initial  set  of  11. 
Subsequent  analysis  led  to  elimination  of  the  three  worst  performing  CODEC's  of 
the  11.  Two  additional  CODEC's  were  eliminated  from  further  consideration  due  to 
limited  functional  capability.  These  CODEC's  were  designated  B  (2.4  kbps)  and  D 
(4.8  kbps).  Neither  could  operate  at  BER's  worse  than  1x10--*. 

In  the  testing,  test  listeners  scored  entire  script  sets  of  test  messages  (either 
Script  1  or  2)  as  processed  through  a  CODEC.  The  script-CODEC  sets  of  messages 
were  presented  to  groups  of  four  listeners  at  a  time.  The  groups  of  non-ATC 
listeners  scored  two  sets  of  script-CODEC  messages.  The  test  facilities  are 
described  in  appendix  C.  A  total  of  24  script-CODEC  sets  were  evaluated  using 
11  CODEC's  plus  the  clear  voice.  Each  CODEC  was  evaluated  with  both  Script  1 
and  2.  ATC  listeners  evaluated  each  script-CODEC  set,  each  group  of  four  ATC 
personnel  evaluating  three  such  sets . 

At  the  completion  of  the  testing  the  tests  were  examined,  and  the  results  entered 
into  a  data  base.  The  use  of  a  computerized  data  base  permitted  simplified 
calculation  of  each  CODEC's  score  with  presentations  catagorized  by  background 
noise  and  BER  conditions. 

3.  TEST  RESULTS. 

3.1  NON-ATC  RESULTS. 

The  intelligibility  test  results  for  the  non-ATC  listeners  are  plotted  in 
figure  1.  This  figure  shows  the  percentage  of  incorrect  responses  for  four 
listeners  of  Script  1  and  four  listeners  of  Script  2  averaged  together.  The 
result  for  the  clear  voice  condition  are  also  included  in  figure  1.  These 
results  average  together  all  of  the  nine  combinations  of  BER  and  background  noise 
condit3r  ns . 

The  results  for  each  CODEC  under  the  nine  conditions  of  background  noise  and  BER 
are  provided  in  table  2.  It  should  be  noted  that  clear  voice  was  not  subjected 
to  BER  conditions.  From  this  data,  it  is  apparent  that  CODEC's  H,  I,  and  C  are 
the  worst  performers,  especially  at  the  higher  BER  and  background  noise 
conditions.  Therefore,  CODEC's  H,  I,  and  C  were  eliminated  from  further  testing, 
leaving  six  CODEC's  for  testing  by  ATC  personnel. 
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TABLE  1.  CODEC  DATA  RATES  AND  ALGORITHMS 


CODEC 

DATA  RATE 
(bps) 

ALGORITHM 

A 

4800 

STC 

B 

2400 

LPC 

C 

2400 

APC 

D 

4800 

LPC 

E 

4800 

SELP 

F 

-  CLEAR 

VOICE 

:  - 

G 

2400 

LPC 

H 

2400 

HDLPC 

I 

4800 

APC 

* 

J 

9600 

SELP 

K 

2400 

STC 

M 

4800 

STC 

Algorithms: 

STC 

LPC 

APC 

SELP 

HDLPC 

Sinusoidal  Transform  Coding 
-  Linear  Predictive  Coding 

Adaptive  Predictive  Coding 

Stochasticly  Excited  Linear  Prediction 
High  Definition  Linear  Predictive  Coding 

*Note:  This  CODEC  was  evaluated  by  the  AEEC  but  was  not  selected 

for  APC  use. 
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PERCENT  OF  ERRORS 


AVERAGE  CODEC  INTELLIGIBILITY  SCORE 


USING  OFFICE  STAFF  AS  TEST  LISTENERS 


CODEC  BY  REFERENCE  LETTER 


FIGURE  1.  NON- AT C  INTELLIGIBILITY  RESULTS 
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TABLE  2.  NON -AT C  INTELLIGIBILITY  RESULTS 


Background  -> 

BER  -> 

QUIET 

10'5  10'3 

10'2 

COCKPIT 

10*5  10'3  10'2 

10 

ATC 

5  10'3 

10‘2 

CODEC 

ID  bos 

% 

of  Errors 

%  of  Errors 

%  of  Errors 

F  (Clear) 

2.5 

0.8 

0.8 

1.7 

5.8 

1.7 

0.0 

3.3 

0.0 

J  (9600) 

0.8 

3.3 

4.2 

5.8 

6.7 

12.5 

3.2 

9.2 

6.1 

A  (4800) 

1.7 

5.0 

6.7 

11.7 

7.5 

10.8 

5.6 

11.7 

14.8 

M  (4800) 

7.3 

5.2 

11.5 

9.4 

10.4 

10.4 

5.0 

12.5 

5.4 

E  (4800) 

6.9 

4.2 

5.6 

10.2 

11.1 

11.6 

9.8 

11.1 

11.1 

G  (2400) 

7.5 

5.8 

8.3 

15.8 

9.2 

19.2 

8.8 

11.7 

14.8 

K  (2400) 

7.5 

9.2 

6.7 

10.8 

18.3 

14.2 

5.6 

18.3 

19.1 

H  (2400) 

10.8 

1.7 

5.8 

12.5 

20.8 

16.7 

13.6 

32.5 

22.6 

I  (4800) 

5.0 

2.5 

12.5 

15.8 

23.3 

29.2 

14.4 

20.8 

37.4 

C  (2400) 

5.8 

6.7 

15.0 

22.5 

19.2 

45.8 

18.4 

35.0 

33.0 

Note:  There  was  no  BER  for  the  clear  voice  conditions.  However, 
the  same  test  items  were  used  as  in  the  CODEC  tests  where  BERs  were 
injected. 
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3.2  ATC  RESULTS. 

The  test  results  for  the  ATC  personnel  are  plotted  in  figure  2.  This  figure 
shows  the  average  percentage  of  incorrect  responses  for  all  listeners  for  each 
CODEC.  Sixty  ATC  controllers  participated  in  the  tests.  Each  CODEC  was 
evaluated  by  24  listeners.  The  results  show  that  the  4.8  kbps  CODEC's  are  within 
tenths  of  a  percent  of  the  9.6  kbps  CODEC.  The  results  also  show  that  the  2.4 
kbps  CODEC's  performed  worse  than  the  4.8  kbps  CODEC's.  However,  the  overall 
intelligibility  of  the  2.4  kbps  CODEC's  still  exceeded  88  percent. 

The  test  scores  were  broken  out  by  BER  and  background  noise  for  each  CODEC.  The 
scores  are  provided  in  table  3  and  plotted  in  figure  3.  The  table  and  the  figure 
show  the  expected  trend  that  intelligibility  declines  as  the  CODEC's  are 
subjected  to  combinations  of  high  levels  of  operational  background  noise  and  BER. 
However,  even  under  the  worst  conditions,  the  intelligibility  of  the  9.6  kbps 
CODEC's  and  the  4.8  kbps  CODEC's  remained  high.  The  performance  of  the  2.4  kbps 
CODEC's  worsened  significantly  at  high  noise  and  BER. 

One  apparent  contradiction  that  can  be  seen  in  several  of  the  plots  is 
intelligibility  scores  at  1x10"^  are  slightly  worse  than  the  same  background 
noise  conditions  a-  1x10"^.  This  may  be  due  to  different  test  messages  being 
used  with  the  difterent  sets  of  conditions.  This  is  suggested  by  the  plot  of 
results  for  clear  voice  messages.  As  in  the  non-ATC  test  result  presentation, 
the  plot  has  columns  marked  with  error  rates  of  1x10"^,  etc.,  and  the  clear  voice 
was  not  actually  subjected  to  bit  errors.  The  markings  are  given  only  for 
referencing  the  results  to  other  CODEC  plots.  As  the  clear  voice  plot  indicates, 
the  test  messages  used  for  CODEC  evaluation  at  a  BER  of  1x10 were  slightly  less 
intelligible  than  those  used  at  1x10'^. 

3.3  ATC  OPINION  SURVEY  RESULTS. 

At  the  conclusion  of  each  scoring  section,  each  listener  was  asked  to  complete  a 
questionnaire.  The  questionnaire  focused  on  acceptability  and  comparison  of 
performance  to  existing  communications  systems.  The  results  of  the  questionnaire 
are  presented  in  figure  4  with  the  intelligibility  scores.  The  figure  shows 
that,  in  the  opinion  of  the  controllers,  the  9.6  CODEC  and  the  best  4.8  kbps 
CODEC  were  nearly  equivalent  to  current  communications.  It  must  be  pointed  out 
that  the  controllers  were  listening  to  a  variety  of  rapidly  changing  conditions, 
including  conditions  worse  than  normal  operations.  This  may  have  affected  their 
opinion  scoring. 

Most  of  the  controllers  used  for  the  testing  had  a  majority  of  their  previous 
communications  experience  on  very  high  frequency  (VHF)  systems.  Some  of  the 
Oakland  controllers  also  had  HF  experience.  Those  controllers  who  distinguished 
between  VHF  and  HF  voice  communications  rated  the  CODEC's  considerably  higher 
than  HF,  but  equivalent  to  or  worse  than  VHF. 

The  controllers  also  rated  the  CODEC's  on  acceptability  (shown  in  figure  4).  The 
figure  shows  the  best  4.8  kbps  CODEC  is  considered  as  acceptable  as  the  9.6  kbps 
CODEC.  Again,  it  must  be  noted  that  the  controllers  based  their  opinions  of 
acceptability  on  listening  to  the  test  scripts,  which  contained  rapidly 
fluctuating  conditions,  some  of  which  were  worse  than  normal  operations.  It  is 
also  interesting  to  note  that  the  best  2.4  kbps  CODEC  was  rated  nearly  as  highly 
as  the  worst  4.8  kbps  CODEC.  The  other  2.4  kbps  CODEC's  were  rated  very  poorly. 
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AVERAGE  CODEC  INTELLIGIBILITY  SCORE 


USING  CONTROLLERS  AS  TEST  LISTENERS 


FIGURE  2.  ATC  INTELLIGIBILITY  RESULTS 


TABLE  3.  ATC  INTELLIGIBILITY  RESULTS 


Background  -> 

QUIET 

COCKPIT 

ATC 

BER  -> 

10'5 

10'3 

10'2 

10' 

5  10'3 

10'2 

10 

'5  10*3 

10'2 

CODEC 

ID  bps 

%  of  Errors 

%  of  Errors 

%  of  Errors 

F  (Clear) 

0.3 

1.0 

0.0 

1.0 

4.2 

1.7 

1.3 

2.1 

0.7 

J  (9600) 

1.4 

3.5 

2.8 

3.1 

8.3 

10.1 

4.0 

8.3 

7.6 

E  (4800) 

2.1 

2.4 

4.2 

3.8 

7.8 

8.7 

3.8 

8.2 

8.8 

M  (4800) 

1.4 

3.8 

2.8 

5.2 

10.1 

10.4 

2.3 

9.0 

5.8 

A  (4800) 

2.1 

2.8 

4.5 

8.0 

7.1 

7.6 

2.4 

10.8 

6.4 

K  (2400) 

2.4 

3.8 

4.9 

6.9 

14.6 

13.5 

3.3 

16.7 

13.0 

G  (2400) 

6.9 

5.9 

5.9 

14.9 

11.5 

15.3 

6.0 

17.7 

15.9 

CODEC  E  PERFORMANCE 


PERCENT  ERRORS  -  USING  ATC  PERSONNEL 


CODEC  M  PERFORMANCE 


PERCENT  ERRORS  -  USING  ATC  PERSONNEL 


FIGURE  3.  ATC  INTELLIGIBILITY  SCORES  CATEGORIZED  BY  BACKGROUND  NOISE 
AND  BER  (SHEET  1  OF  2) 
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CODEC  A  PERFORMANCE 


CODEC  K  PERFORMANCE 


CODEC  G  PERFORMANCE 


PERCENT  ERRORS  -  USINC  ATC  PERSONNEL 


FIGURE  3.  ATC  INTELLIGIBILITY  SCORES  CATEGORIZED  BY  BACKGROUND  NOISE 
AND  BER  (SHEET  2  OF  2) 
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QUESTIONNAIRE  RESULTS 
VERSUS  INTELLIGIBILITY  SCORES 


LISTENER  RESPONSES  CONVERTED  TO  PERCENT 
100| - ^ - rr 


INTELLIGIBILITY  COMPARISON  TO  ACCEPTABILITY 
EXISTING  SYSTEMS 


IS G  2.4  □  K  2.4  Ha  4.8  [1M4.8  ^E4.8  ■  J  9.6 

□  clear 


Notes:  1.  Intelligibility  scores  are  shown  as  percent  correct. 

2.  Code  for  "comparison  to  existing"  category: 

100%  —  Much  better  than  current  communications 
75%  —  Better  than  current  communications 
50%  -  Equivalent  to  current  communications 
25%  -  Worse  than  current  communications 
0%  -  Much  worse  than  current  communications 

3.  Acceptability  results  show  percentage  of  ATC  personnel  who  considered 
the  CODEC  under  test  to  be  acceptable  for  ATC. 


FIGURE  4.  ATC  OPINION  RESULTS 
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3.4  COMPARISON  OF  ATC  RESULTS  WITH  NON-ATC  RESULTS. 


Figure  5  compares  the  test  results  of  the  ATC  and  non-ATC  participants.  The 
figure  shows  the  controllers  generally  had  higher  intelligibility  scores  than  the 
non-ATC  group;  however,  the  two  groups  scored  the  CODEC's  similarly  in  terms  of 
relative  performance.  This  agreement  supports  the  use  of  the  non-ATC  scores  to 
eliminate  the  three  worst  performing  CODEC's  as  noted  in  paragraph  3.1. 

3.5  COMPARISON  OF  TEST  SECTION  RESULTS. 

Intelligibility  testing  was  divided  into  three  parts,  representing  three  levels 
of  ''openness."  Part  1  was  a  pseudo-DRT  test,  Part  2  contained  numerals  in 
context,  and  Part  3  contained  isolated  words.  Figure  6  shows  the  percentage  of 
total  ATC  test  errors  broken  out  by  part.  The  figure  shows  the  "closed"  DRT-like 
Part  1  received  approximately  22  percent  of  the  errors,  the  partially  "open"  Part 
2  received  30  percent  of  the  errors,  and  the  more  "open"  Part  3  received  over 
47  percent  of  the  errors.  These  results  agree  with  industry  experience  of  words 
being  more  easily  misinterpreted  in  an  "open"  test  than  a  "closed"  test. 

4.  CONCLUSIONS. 

The  following  conclusions  can  be  drawn  from  the  results  of  the  Phase  II 
coder/decoder  (CODEC)  test  program: 

a.  That  the  intelligibility  of  the  4.8  kilobits  per  second  (kbps)  CODEC's 
is  nearly  as  high  as  the  9.6  CODEC  in  an  air  traffic  control  (A.TC)  environment. 
The  9.6  kbps  CODEC  was  an  improved  version  of  one  of  the  CODEC's  evaluated  in  the 
Airline  Electronics  Engineering  Committee  (AEEC)  selection  of  a  CODEC  for 
Aeronautical  Passenger  Communications  (APC)  use. 

b.  That  the  acceptability  of  the  4.8  kbps  CODEC's  range  from  approximately 
equivalent  to  slightly  worse  than  the  9.6  kbps  CODEC. 

c.  That  the  intelligibility  of  the  4.8  kbps  CODEC's  and  the  9.6  kbps  CODEC 
declined  under  conditions  combining  bit  error  rates  (BER's)  and  operational 
background  noise;  however,  even  under  these  conditions  the  intelligibility 
remained  high  (better  than  89  percent). 

d.  That  the  opinion  of  the  ATC  personnel  was  split  over  the  acceptability 
of  the  9.6  kbps  and  the  4.8  kbps  CODEC's,  with  roughly  half  of  the  controllers 
stating  that  the  CODEC's  were  acceptable  for  ATC  use. 

e.  That  the  intelligibility  of  the  2.4  kbps  CODEC's  was  low,  especially 
under  conditions  of  BER  and  background  noise;  the  acceptability  was  also  low. 

5 .  RECOMMENDATIONS . 

It  is  recommended  that  a  Phase  III  coder/decoder  (CODEC)  test  program  be 
initiated  to: 

a.  Evaluate  the  acceptability  of  available  4.8  kilobits  per  second  (kbps) 
CODEC's  compared  to  standard  air  traffic  control  (ATC)  voice  communications. 

This  includes  very  high  frequency  (VHF) ,  high  frequency  (HF) ,  and  the  9.6  kbps 
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AVERAGE  CODEC  INTELLIGIBILITY  SCORE 


COMPARISON  OF  LISTENER  GROUPS 


POq  OFFICE  STAFF 


CODEC  BY  REFERENCE  LETTER 

555$  CONTROLLERS 


FIGURE  5.  COMPARISON  OF  NON-ATC  AND  ATC  RESULTS 


DISTRIBUTION  OF  ERRORS  BY  TEST  TYPE 

ALL  CODECS  &  SCRIPTS  COMBINED 


FIGURE  6.  DISTRIBUTION  OF  ATC  RESULTS  OVER  SECTION 
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CODEC  which  was  selected  by  Airline  Electronics  Engineering  Committee  (AEEC)  for 
Aeronautical  Passenger  Communications  (APC)  use. 


b.  Develop  a  test  and  criteria  for  selecting  a  4.8  kbps  CODEC  for  ATC  use. 
This  selection  test  would  be  based,  in  part,  on  an  objective  intelligibility  test 
(similar  to  Phase  II  testing)  and  a  subjective  acceptability  test. 

Additionally,  it  is  recommended: 

a.  That  the  test  developed  in  the  Phase  III  program  be  used  to  select  a 
4 . 8  kbps  CODEC  for  ATC  use . 

b.  That  the  algorithm  of  the  selected  4.8  kbps  CODEC  be  proposed  for 
worldwide  standardization  for  ATC  use. 
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APPENDIX  A  -  TEST  SHEETS 


This  appendix  contains  the  test  sheets  and  the  correct  answers  used  for  Script  1 
and  Script  2  of  the  Phase  II  intelligibility  test. 


A-l 


FAA  PHASE  2  SUBJECTIVE  LISTENING  EVALUATION 


NAME  : 

GROUP/SECTION  : 
DATE  : 

TIME  : 


YEARS  EXPERIENCE  : 


TYPE  EXPERIENCE  : 


AGE  (circle  one)  : 

20-24 

25-29 

30-34 

35-39 

40-44 

45-49 

50-54 

>55 


A-2 


PART  1  :  WORD  PAIR  TFST 


1) 

SECTOR  _ 

VECTOR  _ 

-  13) 

2) 

BRAIN  _ 

RAIN  _ 

-  14) 

3) 

SIX  _ 

TRICKS  _ 

_  15) 

4) 

SEVEN  _ 

HEAVEN  _ 

-  16) 

5) 

ROCK 

MACH  _ 

-  17) 

6) 

REPORT  _ 

RESORT  _ 

-  18) 

7) 

FLIGHT 

HEIGHT  _ 

19) 

8) 

FOUR 

FIVE  _ 

_  20) 

9) 

ONE  _ 

FUN  _ 

21) 

10) 

RIGHT  _ 

LIGHT  _ 

-  22) 

11) 

TWO  _ 

THREE  _ 

23) 

12) 

WEST  _ 

TEST  _ 

24) 

ntnp. 

25) 

WINE 

ALTTinnp 

26) 

ATTTTnnr 

sow 

27) 

SOTTTB 

ZERO 

28) 

HERO 

TURN 

29) 

BURN 

CENTER  _____ 

30) 

ENTER  - 

TREE  _ 

31) 

THREE  -- 

PEACH  ___ 

32) 

REACH  _ 

RUNWAY  __ 

33) 

SUNDAY  __ _ 

LEAR 

34) 

CLEAR 

SMTT.ES 

35) 

MTT.ES 

end 

36) 

SEND 

RHYME 

CLIMB 

DIVE 

FIVE 

DATE 

EIGHT 

HELL 

HAIL 

DANGER 

RANGER 

STOP 

DROP 

FIRE 

HIGHER 

DAMPER 

DANDER 

SLOW 

FLOW 

FRONT 

RUNT 

CLOUDS 

CROWDS 

INTERCEPT 

INCORRECT 
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PART  2:  FILL  IN  THE  BLANK  WITH  «'S  TEST 


01. 
02  . 
03. 
04. 
05. 
06. 
07. 
08. 
09. 
10. 
11. 
12. 
13  . 

14. 

15. 

16. 

17. 

18. 

19. 

20. 
21. 
22. 

23. 

24. 

25. 

26. 

27. 

28. 

29. 

30. 

31. 

32. 
33  . 

34. 

35. 

36. 


Continental  _ 

Interstate  _ 

This  is  Falcon 
Power 


_,  climb  and  maintain  FL  290. 

Roger. 

_  Charlie  Delta  at  flight  level  27C 


Tango  Aztec 


_ ,  say  again. 

Alpha,  how  do  you  read? 

Intercept  the  Deer  Park  _  radial. 

Leaving  flight  level  _  for  290. 

Bonanza  _  Golf,  Roger. 

Engine  failure  on  Northwest  _  . 

Eastern  731  contact  Atlanta  Center  on 

Hold  southwest  on  the  Hampton  _ 

Expect  further  clearance  at  _ 


radial . 


Climb  and  maintain  flight  level 

Talan  3  reaching  flight  level  _ 

Any  ride  reports  at  flight  level 

Cleared  to  land  runway  _ . 

Bearcat  _ 

We  are  VFR  at 

Piedmont  _ 

Delta 


southbound? 


left. 


Yankee  Romeo,  Roger. 


cleared  direct  Boston. 


at  flight  level  330. 


Follow  the  American  _  . 

Contact  New  York  Center  on  _  . 

Beechcraft  _  Romeo  Lima  to  flight  level  310. 


Intercept  Jay 


on  your  present  heading. 


Reaching  and  maintaining  flight  level  _  . 

This  is  Northwest  _  ,  out  of  FL  270  for  FL  280. 

Baron  _ 

United 


Hotel,  requesting  traffic  advisories, 
have  a  good  day. 


Request  flight  level  _  . 

United  60,  climb  and  maintain  flight  level 

Contact  San  Juan  on  _  . 

Contact  New  York  Center  _  . 

Cherokee  _  Alpha  over  Kansas  City. 


This  is  November  _ 

Increase  to  mach  point 
Traffic  3  o'clock  _ 


Romeo. 


miles  southbound. 
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PART  4 :  QUESTIONNAIRE 


1.  Overall,  would  you  say  that  what  you  have  just  listened  to  is 
(circle  one) 

a.  much  better  than 

b.  slightly  better  than 

c.  about  the  same  as 

d.  slightly  worse  than 

e.  much  worse  than 

your  present  form  of  communications. 


2.  Overall,  would  this  form  of  communications  be  acceptable  to 
you  in  your  present  environment  (circle  one)? 

a.  yes 

b.  no 


3.  Did  the  presence  of  background  noise  affect  your  answer  in 
question  2  (circle  one)? 

a.  yes 

b.  no 


4.  Please  comment  on  the  similarities  and/or  differences  between 
what  you  have  just  heard  and  your  present  form  of 
communications . 


5.  Any  further  comments  on  this  evaluation? 
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SOLUTIONS  TO  SECTION  1 
PHASE  2  CODEC  TEST 


Solutions  to  ScriDt  1  Solutions  to  Scnot  2 


01. 

2/1 

01. 

2/1 

02. 

1/2 

02. 

1/2 

03. 

1/2 

03. 

2/1 

04. 

2/1 

04. 

1/2 

05. 

2/1 

05. 

2/1 

06. 

2/1 

06. 

2/1 

07. 

2/1 

07. 

1/2 

08. 

1/2 

08. 

1/2 

09. 

2/1 

09. 

2/1 

10. 

1/2 

10. 

1/2 

11. 

2/1 

11. 

2/1 

12. 

1/2 

12. 

2/1 

13. 

1/2 

1 3» 

1/2 

14. 

1/2 

14. 

1/2 

15. 

2/1 

15. 

2/1 

16. 

2/1 

16. 

2/1 

17. 

1/2 

17. 

1/2 

18. 

2/1 

18. 

2/1 

19. 

1/2 

19. 

2/1 

20. 

1/2 

20. 

2/1 

21. 

1/2 

21. 

1/2 

22. 

2/1 

22. 

1/2 

23. 

1/2 

23. 

2/1 

24. 

2/1 

24. 

1/2 

25. 

1/2 

25. 

1/2 

26. 

1/2 

26. 

2/1 

27. 

2/1 

27. 

1/2 

28. 

1/2 

28. 

2/1 

29. 

2/1 

29. 

2/1 

30. 

2/1 

30. 

1/2 

31. 

2/1 

31. 

1/2 

32. 

2/1 

32. 

2/1 

33. 

2/1 

33. 

1/2 

34. 

1/2 

34. 

2/1 

35. 

1/2 

35. 

1/2 

36. 

1/2 

36. 

1/2 
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SOLUTIONS  TO  SECTION  2 
PHASE  2  CODEC  TEST 


Solutions  to  Script  1 


Solutions  to  Script  2 


01. 

125 

01. 

152 

02. 

953 

02. 

.955 

03. 

41 

03. 

51 

04. 

98 

04. 

89 

05. 

4723 

05. 

4723 

06. 

221 

06. 

212 

07. 

390 

07. 

360 

08. 

8231 

08. 

8123 

09. 

34 

09. 

43 

10. 

1 33.45 

10. 

123.45 

11. 

264 

11. 

462 

12. 

2040 

12. 

2050 

13. 

260 

13. 

270 

14. 

380 

14. 

320  ‘ 

15. 

250 

15. 

240 

16. 

24 

16. 

42 

17. 

5 

17. 

6 

18. 

17500 

18. 

16500 

19. 

953 

19. 

552 

20. 

845 

20. 

435 

21. 

747 

21. 

727 

22. 

134.5 

22. 

134.6 

23. 

759 

23. 

781 

24. 

121 

24. 

212 

25. 

330 

25. 

220 

26. 

323 

26. 

233 

27. 

4365 

27. 

4365 

28. 

8 

28. 

2 

29. 

280 

29. 

270 

30. 

350 

30. 

340 

31. 

125.8 

31. 

124.8 

32. 

128.3 

32. 

132.8 

33. 

351 

33. 

315 

34. 

6731 

34. 

6771 

35. 

78 

35. 

75 

36. 

20 

36. 

20 
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APPENDIX  B  -  TEST  ITEM  CONDITIONS 


This  appendix  contains  the  list  of  conditions  for  each  test  item  on  each  script. 
For  example,  the  first  entry  on  the  list  for  Script  1  is  for  Scenario  22.  The 
scenario  number  corresponds  to  a  test  item  for  each  section  of  the  test.  By 
referring  to  the  test  in  appendix  A,  the  test  item  for  Part  1,  Number  22,  is  the 
word  pair  "lear"  and  "clear."  Referring  again  to  the  list  of  conditions,  it  is 
indicated  that  this  word  pair  was  spoken  by  a  female  with  quiet  background 
condition,  and  that  a  BER  of  1x10'^  was  introduced.  The  BER  was  adjusted  for 
each  section  of  the  test,  as  is  shown  in  the  last  column. 
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TABLE  B— 1.1.  TEST  CONDITIONS  FOR  EACH  ITEM  ON  SCRIPT  1 


SCENARIO 

VOICE 

GENDER 

BACKGROUND 

NOISE 

VOICE 

ERROR 

TYPE 

section 

1 

22 

FEMALE  1 

QUIET 

Kga 

Da 

04 

FEMALE  2 

QUIET 

in 

m 

US 

11 

FEMALE  3 

QUIET 

— 

IE-5 

IE-3 

IE-2 

16 

FEMALE  4 

QUIET 

— 

IE-3 

IE-2 

IE-3 

13 

MALE  1 

QUIET 

— 

IE-3 

IE-2 

IE-3 

28 

MALE  2 

QUIET 

— 

IE-2 

IE-5 

IE-5 

33 

MALE  3 

QUIET 

— 

IE-2 

IE-5 

IE-5 

07 

MALE  4 

QUIET 

— 

IE-5 

IE-3 

IE-2 

25 

MALES 

QUIET 

— 

IE-2 

IE-5 

IE-5 

21 

MALES 

QUIET 

— 

IE-3 

IE-2 

IE-3 

02 

MALE  7 

QUIET 

— 

IE-5 

IE-3 

IE-2 

35 

MALES 

QUIET 

— 

IE-2 

IE-5 

IE-5 

01 

FEMALE  1 

ATC  ROOM 

ATC 

IE-5 

IE-3 

IE-2 

12 

FEMALE  2 

ATCROOM 

ATC 

IE-5 

IE-3 

IE-2 

06 

FEMALE  3 

ATC  ROOM 

ATC 

IE-5 

IE-3 

IE-2 

24 

FEMALE  4 

ATCROOM 

ATC 

IE-3 

IE-2 

IE-3 

10 

MALE  1 

ATCROOM 

ATC 

IE-5 

IE-3 

IE-2 

31 

MALE  2 

ATCROOM 

ATC 

IE-2 

IE-5 

IE-5 

30 

MALE  3 

ATCROOM 

ATC 

IE-2 

IE-5 

IE-5 

08 

MALE  4 

ATCROOM 

ATC 

IE-5 

IE-3 

IE-2 

19 

MALE  5 

ATCROOM 

ATC 

IE-3 

IE-2 

IE-3 

23 

MALE  6 

ATC  ROOM 

ATC 

IE-3 

IE-2 

IE-3 

36 

MALE  7 

ATC  ROOM 

ATC 

IE-2 

IE-5 

IE-5 

32 

MALES 

ATC  ROOM 

ATC 

IE-2 

IE-5 

IE-5 

26 

FEMALE  1 

COCKPIT 

PILOT 

IE-2 

IE-5 

IE-5 

29 

FEMALE  2 

COCKPIT 

PILOT 

IE-2 

IE-5 

IE-5 

09 

FEMALE  3 

COCKPIT 

PILOT 

IE-5 

IE-3 

IE-2 

18 

FEMALE  4 

COCKPIT 

PILOT 

IE-3 

IE-2 

IE-3 

17 

MALE  1 

COCKPIT 

PILOT 

IE-3 

IE-2 

IE-3 

15 

MALE  2 

COCKPIT 

PILOT 

IE-3 

IE-2 

IE-3 

03 

MALE  3 

COCKPIT 

PILOT 

IE-5 

IE-3 

IE-2 

34 

MALE  4 

COCKPIT 

PILOT 

IE-2 

IE-5 

IE-5 

14 

MALES 

COCKPIT 

PILOT 

IE-3 

IE-2 

IE-3 

20 

MALE  6 

COCKPIT 

PILOT 

IE-3 

IE-2 

IE-3 

27 

MALE  7 

COCKPIT 

PILOT 

IE-2 

IE-5 

IE-5 

05 

MALES 

COCKPIT 

PILOT 

IE-5 

IE-3 

IE-2 
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TABLE  B-1.2.  TEST  CONDITIONS  FOR  EACH  ITEM  ON  SCRIPT  2 


VOICE 

TYPE 


SCENARIO 

VOICE 

BACKGROUND 

GENDER 

NOISE 

FEMALE  1 
FEMALE  2 
FEMALE  3 
FEMALE  4 
MALE  1 
MALE  2 
MALE  3 
MALE  4 
MALE  5 
MALES 
MALE  7 
MALE  8 
FEMALE  1 
FEMALE  2 
FEMALE  3 
FEMALE  4 
MALE  1 
MALE  2 
MALE  3 
MALE  4 
MALE  5 
MALE  6 
MALE  7 
MALE  8 
FEMALE  1 
FEMALE  2 
FEMALE  3 
FEMALE  4 
MALE  1 
MALE  2 
MALE  3 
MALE  4 
MALES 
MALES 
MALE? 
MALES 


QUIET 

QUIET 

QUIET 

QUIET 

QUIET 

QUIET 

QUIET 

QUIET 

QUIET 

QUIET 

QUIET 

QUIET 

ATCROOM 

ATCROOM 

ATCROOM 

ATCROOM 

ATCROOM 

ATCROOM 

ATCROOM 

ATCROOM 

ATCROOM 

ATCROOM 

ATCROOM 

ATCROOM 

COCKPIT 

COCKPIT 

COCKPIT 

COCKPIT 

COCKPIT 

COCKPIT 

COCKPIT 

COCKPIT 

COCKPIT 

COCKPIT 

COCKPIT 

COCKPIT 


ATC 

ATC 

ATC 

ATC 

ATC 

ATC 

ATC 

ATC 

ATC 

ATC 

ATC 

ATC 

PILOT 

PILOT 

PILOT 

PILOT 

PILOT 

PILOT 

PILOT 

PILOT 

PILOT 

PILOT 

PILOT 

PILOT 


ERROR 

section 

1 

section 

2 

section 

3 

IE-2 

IE-5 

Kgfl 

IE-5 

IE-3 

H a 

IE-5 

IE-3 

IE-2 

IE-2 

IE-5 

IE-5 

IE-2 

IE-5 

IE-5 

IE-3 

IE-2 

IE-3 

IE  3 

IE-2 

IE-3 

IE-5 

IE-3 

IE-2 

IE-3 

IE-2 

IE-3 

IE-2 

IE-5 

IE-5 

IE-5 

IE-3 

IE-2 

IE-3 

IE-2 

IE-3 

IE-5 

IE-3 

IE-2 

IE-5 

IE-3 

IE-2 

IE-5 

IE-3 

IE-2 

IE-2 

IE-5 

IE-5 

IE-5 

IE-3 

IE-2 

IE-3 

IE-2 

IE-3 

IE-3 

IE-2 

IE-3 

IE-5 

IE-3 

IE-2 

IE-2 

IE-5 

IE-5 

IE-2 

IE-5 

IE-5 

IE-3 

IE-2 

IE-3 

IE-3 

IE-2 

IE-3 

IE-3 

IE-2 

IE-3 

IE-3 

IE-2 

IE-3 

IE-5 

IE-3 

IE-2 

IE-2 

IE-5 

IE-5 

IE-2 

IE-5 

IE-5 

IE-2 

IE-5 

IE-5 

IE-5 

IE-3 

IE-2 

IE-3 

IE-2 

IE-3 

IE-2 

IE-5 

IE-5 

IE-2 

IE-5 

IE-5 

IE-3 

IE-2 

IE-3 

IE-5 

IE-3 

IE-2 

APPENDIX  C  -  TEST  FACILITIES  FOR  RECORDING  AND  LISTENING 


This  appendix  provides  a  description  of  the  test  setups  used  in  Phase  II  for 
generating  the  test  tapes  and  for  conducting  the  listening  tests. 

Figure  C-l.l  shows  the  equipment  setup  for  the  test  tape  generation.  The 
unprocessed  master  tape  contains  recordings  of  the  test  scripts  in  the 
appropriate  background  environments.  The  recording  is  first  played  through  the 
TAS  1010  phone  line  simulator,  which  was  configured  to  simulate  an  "Average  Bell" 
2-wire  circuit. 

The  analog  output  of  the  phone  line  simulator  was  then  converted  by  the  transmit 
CODEC  into  a  digital  bit  stream,  which  was  input  into  the  TTC  1020B  Digital  Error 
and  Delay  simulators.  These  simulators  injected  the  appropriate  BER  and 
introduced  the  270  ms  delay  representing  the  satellite  link.  The  digital  bit 
stream  was  converted  to  an  analog  signal  by  the  receive  CODEC,  which  was 
recorded. 

One  tape  was  recorded  for  each  CODEC  on  each  script.  These  tapes  were  then  used 
to  conduct  the  Phase  II  tests. 

Figure  C-1.2  shows  the  setup  for  the  listening  tests.  Each  CODEC  tape  for  each 
script  was  played  for  four  listeners  simultaneously;  however,  each  listener  could 
select  the  listening  level  they  preferred. 
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UNPROCESSED  MASTER 
TEST  TAPE 


PROCESSED 
TEST  TAPE 


HS-232C 


NOTE:  CODECs  A,  K,  &  M  were  configured  without  the  delay 

simulator.  CODEC  H  was  configured  without  the  error  and 
delay  simulator. 

FIGURE  C-l.l.  PHASE  II  TAPE  GENERATION  EQUIPMENT  SETUP 
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FIGURE  C-1.2.  PHASE  II  LISTENING  TEST  SETUP 
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